Fast Inference in Phrase Extraction Models with Belief Propagation
نویسندگان
چکیده
Modeling overlapping phrases in an alignment model can improve alignment quality but comes with a high inference cost. For example, the model of DeNero and Klein (2010) uses an ITG constraint and beam-based Viterbi decoding for tractability, but is still slow. We first show that their model can be approximated using structured belief propagation, with a gain in alignment quality stemming from the use of marginals in decoding. We then consider a more flexible, non-ITG matching constraint which is less efficient for exact inference but more efficient for BP. With this new constraint, we achieve a relative error reduction of 40% in F5 and a 5.5x speed-up.
منابع مشابه
Belief Propagation with Strings
Strings and string operations are very widely used, particularly in applications that involve text, speech or sequences. Yet the vast majority of probabilistic models contain only numerical random variables, not strings. In this paper, we show how belief propagation can be applied to do inference in models with string random variables which use common string operations like concatenation, find/...
متن کاملStructured Belief Propagation for NLP
Statistical natural language processing relies on probabilistic models of linguistic structure. More complex models can help capture our intuitions about language, by adding linguistically meaningful interactions and latent variables. However, inference and learning in the models we want often poses a serious computational challenge. Belief propagation (BP) and its variants provide an attractiv...
متن کاملChoosing a Variable to Clamp: Approximate Inference Using Conditioned Belief Propagation
In this paper we propose an algorithm for approximate inference on graphical models based on belief propagation (BP). Our algorithm is an approximate version of Cutset Conditioning, in which a subset of variables is instantiated to make the rest of the graph singly connected. We relax the constraint of single-connectedness, and select variables one at a time for conditioning, running belief pro...
متن کاملLagrangian Relaxation for Inference in Natural Language Processing
There has been a long history in combinatorial optimization of methods that exploit structure in complex problems, using methods such as dual decomposition or Lagrangian relaxation. These methods leverage the observation that complex inference problems can often be decomposed into efficiently solvable sub-problems. Thus far, however, these methods are not widely used in NLP. In this talk I will...
متن کاملRegion Extraction Based on Belief Propagation for Gaussian Model
We show a fast algorithm for region extraction based on belief propagation with loopy networks. The solution to this region segmentation problem, which includes the region extraction problem, is of significant computational cost if a conventional iterative approach or statistical sampling methods are applied. In the proposed approach, Gaussian loopy belief propagation is applied to a continuous...
متن کامل